A hierarchical approach to designing approximate reasoning-based controllers for dynamic physical systems

This paper presents a new technique for the design of approximate reasoning-based controllers for dy­ namic physical systems with interacting goals. In this approach, goals are achieved based on a hier­ archy defined by a control knowledge base and re­ main highly interactive during the execution of the control task. The approach has been implemented in a rule-based computer program which is used in conjunction with a prototype hardware system to solve the cart-pole balancing problem in real­ time. It provides a complementary approach to the conventional analytical control methodology, and is of substantial use where a precise mathe­ matical model of the process being controlled is not available. Introduction and Motivation Expert human controllers often perform superbly un­ der conditions of uncertainty and imprecision using mainly approximate reasoning. They select control ac­ tions based on a quick assessment of the process which they are controlling. Control theorists have success­ fully dealt with a large class of control problems by mathematically modeling the process and solving these analytical models to generate control actions. How­ ever, the analytical models tend to become complex and infeasible to use, especially for large, intricate sys­ tems. The non-linear behavior of many practical sys­ tems makes the analytical approach even more diffi­ cult, sometimes impossible. Starting with Mamdani and Assilian[Mamdani 75], who based their work on Zadeh's pioneering work on fuzzy set theory [Zadeh 65], an alternative method in •sterling Federal Sy stems •Dept. of Electrical Engineering, National Taiwan Uni­ versity, Taipei, Taiwan, R.O.C. toept. of Electrical Engineering and Computer Science, University of California, Berkeley, 94720 iDept. of Electrical Engineering and Computer Science, University of California, Berkeley, 94720 'ISRO Satellite Center, Banglore 560017, India design of controllers has been proposed. This tech­ nique, generally known as fuzzy control, has experi­ enced much success, especially in recent years. These controllers mimic the performance of human expert operators by encoding their knowledge in terms of linguistic control rules which may contain fuzzy la­ bels (e.g., HOT, MEDIUM, SMALL). Among the successful applications of this theory are the guid­ ance control of subway trains in the city of Sendai in Japan [Yasunobu 85] and cement kiln control [Ostergaard 77]. A recent survey of this field has been provided by [Lee 90b]. In general, these controllers have been especially effective for systems with a single goal. In this paper, we provide a new method for de­ signing approximate reasoning-based controllers which can achieve conjunctive and interacting goals. We com­ pare our method with the state feedback control, a popular approach in modern digital control. This comparative study is made using computer simulation and a hardware implementation of a cart­ pole balancing system which represents a typical non­ linear system. This interesting problem has served as a basis for study by many connectionist works (e.g., [Widrow 87]) and control theorists (e.g., [Shaefer 66]). Learning of the control process for pole balancing has been studied by Michie and Chambers [Michie 68], Sel­ fridge, Sutton, and Barto [Selfridge 85], [Barto 83], and by Lee [Lee 90a,Lee &Berenji 89]. In this learning research, the objective has been to write a program which can learn to keep the pole balanced. The organization of this paper is as follows. With a brief overview of approximate reasoning-based control, we introduce a new method for designing controllers for dynamic physical systems. We then apply this method to the cart-pole balancing problem. The results of our simulations and hardware tests are discussed next. Fi­ nally, we contrast the performance of a controller based on our new approach with a conventional analytical controller. Approximate Reasoning-Based Controllers A difficulty in employing AI techniques in real-time control is how to handle impreci&ion in the knowl­ edge expressed by human expert operators. Fuzzy set theory provides a facility to express the imprecise knowledge by using lingui&tic variable3 [Zadeh 75]. We have argued elsewhere about the importance of han­ dling different types of uncertainty in AI systems (e.g., [Berenji 88a], [Berenji 88b]). The basic idea in fuzzy control centers around the labeling process, in which the reading of a sensor is translated into a label as done by human operators. For example, in the context of controlling a nuclear reactor [Bernard 88], an observed reactor period (i.e., the rate of rise of the power) might be classified as too 3hort, 3hort, or negative. It is important to note that the transition between the labels are continuous rather than abrupt. This means that a reactor's period of 90 seconds might be termed too 3hort to degree 0.2, 3hort to degree 1.0, and negative to degree 0.0 [Bernard 88]. A similar concept is used in our experiment: an angu­ lar position of say 5 degrees might be called Po3itive to a degree of .8 and Zero (i.e., a label used to describe very small angles) to degree of 0.2. This idea of par­ tial matching plays an important role in fuzzy control, and is related to the concept of a membership function u�d in fuzzy set theory where the boundary of a set is not sharp and the degree of member3hip specifies how strongly an element belongs to a set. The knowledge base of an approximate reasoning­ based controller is a collection of lingui&tic con­ trol rule3 which are described using lingui&tic variable3[Zadeh 75]. For example, IF X is A and Y is B THEN Z is C is a linguistic control rule where X and Y are sen­ sor readings from the plant and Z corresponds to the output (i.e., the recommended action). A, B, and C are linguistic values such as LARGE, POSITIVE, etc. which are represented by membership functions ( usu­ ally in triangular or trapezoidal forms). When partic­ ular values for X and Y are sensed, then these val­ ues are matched against the membership functions of A and B respectively. As a result of this matching, the degree that each precondition is satisfied will be known. Since sensor readings usually trigger several control rules at the same time, a conflict re3olution strategy is needed. A Maz-Min compo3itional rule of inference, as explained below, is commonly used. Assume that we have the following two rules: Rule 1: IF X is At and Y is Bt THEN Z is Ct Rule 2: IF X is A2 and Y is B2 THEN Z is C2 Now, if we have Zt and Yt as the sensor readings for fuzzy variables X and Y, then their truth value3 are represented by �A1 (zt) and �B1 (Yt) respectively for 363 Rule 1, where J.'A1 represents the membership function for A1. Similarly for Rule 2, we have J.'A2(zt) and �B, (yt) as the truth values of the preconditions. Then the 3trength of Rule 1 can be calculated by: a1 = �A1 (zt) A �B1 (yl). Similarly for Rule 2: a2 = �A,(zt) A �s,(Yt)· The effect of the strength of Rule 1 on its conclusion is calculated by: �c� (w) = a1 A �c1 (w), and for Rule 2: �c�(w) = a2 A �c,(w). This means that as a result of reading sensor values z1 and Yt1 Rule 1 is recommending a control action with �c� ( w) as its membership function and Rule 2 is recommending a control action with l'c; ( w) as its membership function. The conflict-resolution process then produces �c(w) = �c�(w)V�q(w) = [atA�c1(w)]v[a2A�c,(w)] where �c(w) is a pointwise membership function for the combined conclusion of Rule 1 and Rule 2. The A and V operators in above are defined to be the min and max functions respectiveiy [Marudani 75]. The re­ sult of this last operation (J.'c(w)) has to be translated ( defuzzifiet!) to a single value. This necessary opera­ tion produces a nonfuzzy control action that best rep­ resents the membership function of an inferred fuzzy control action. The Center Of Area ·(COA) method (see [Lee 90b]) can be used here. Assuming a discrete universe, we have z· Ei=t w; * �c(w;) Ej=l �c(w;) where n is the number of quantization levels of the output. Hierarchical Control and Conjunctive Goal Achievement In this section we develop a method for designing con­ trollers which (a) obey a hierarchical process in fo­ cusing attention on a particular goal at each time instance, and (b) can achieve interacting goals si­ multaneously. This discussion is related in many ways to recent AI planning research where integrated planning-execution-control architectures are being ex­ plored (e.g., [Drummond 89,Bresina 90]). In this sec­ tion, we present a brief discussion of our method in the general context of approximate reasoning and in Sec­ tion , we demonstrate the use of this technique in the domain of cart-pole balancing. The method includes the following steps: I